Map Incident Priority to Specific Actions

#1

My team is currently working on setting up various integrations with PagerDuty such as initiating a PagerDuty incident when a CloudWatch alert is issued. At the same time, automatically creating a Jira ticket and sending a notification to a Slack channel. While working well, we would like to prioritize these incidents and notify others differently based on the severity.

I know that PagerDuty can be set up to have services with High and Low urgency. Therefore, if one of those is triggered, certain types of notifications or escalations can take place. However, can this be based off of Incident Priority? For example, if a P1 is selected when opening an incident, can this trigger an escalation policy, but if a P5 is selected, just open a Jira ticket using the associated integration? We are trying to figure out a way to map incident priority to different types of actions. Is this currently possible?

Also, is it possible to set incident priority when using a service integration like CloudWatch?

Thanks.

#2

Hi Scott!

At this time, there isn’t a straight-forward way in creating different event rules for one service based on the priority status.

The best approach for what you’re looking for would be to create a service for each priority instance notification group and route events from your JIRA instance to the applicable PagerDuty service.

This would then allow you to categorize the various AWS Cloudwatch alarms using the integration guide to follow the service’s notification system.

#3

Hi Geeth,

Are you saying that to accomplish what we want to do now we should create multiple services (one for each incident priority) and set up the notifications and integrations for each of those?

Also, I am a little confused regarding the statement about categorizing the CloudWatch alarms. Do you mean that for certain alarms, we direct them to the services associated with their predetermined priorities?

Thanks again for your assistance.

Scott

#5

Hi Scott,

Are you saying that to accomplish what we want to do now we should create multiple services (one for each incident priority) and set up the notifications and integrations for each of those?

No, that isn’t going to be necessary! I’m curious about something which I don’t want to guess at. Which is true of your current workflow:

  • Monitoring software picks up issue. Passes that on to JIRA and opens a ticket. JIRA hands off to PagerDuty which wakes everybody up and puts a message in Slack.

  • Monitoring software picks up an issue. Passes issue to PagerDuty. PD passes that to JIRA which opens a ticket.

In your initial description it sounds as if you’re doing the latter but I really like to be certain about that because those little subtleties can change how to think toward a solution.

Would it be possible for us to have a call tomorrow? Probably not more than 15 or 30 minutes I think. It’s very easy to get wrapped around the axle with this nomenclature so I’d like to be able to talk in real time for a few just so I know I’m answering the question you actually asked and not the one I thought you asked.

Cheers
AJ

#6

Hi AJ,

The second scenario kind of describes our situation. Our monitoring solution passes an issue detected along to Pager Duty. We currently are leveraging JIRA Cloud, and even though we have PD & JIRA integrated, it is my understanding that at this time a JIRA Cloud ticket cannot be automatically created. Therefore, we have to rely on manually pressing the button in the PD console to initiate the JIRA ticket.

Anyhow, we can definitely have a conversation and try to figure out the best way to accomplish what we are wanting to do. Just let me know when you would like to meet.

Thanks again.

Scott